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ABSTRACT 

A study conducted in Israel investigated the 
relevance of subject-specific reading passages to performance on 
reading comprehension tests for advanced university students of 
English as a second language. The research specifically examined (1) 
whether students performed better when the reading test content was 
directly related to their field of study or when the content was in 
another field but of comparable difficulty and comprehensible to the 
educated layman, and (2) whether the position of a text in a 
multiple-text exam affects reading comprehension performance. Results 
showed that comprehension was affected in some cases by content, but 
less than expected, a finding possibly explained by individuals* 
English reading competence or by the broad groupings of fields of 
study (science and technology, humanities, and biology). The order of 
presentation did not seem to affect performance, which has 
implications for situations in which different forms of the same test 
are desirable. (MSE) 
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DO CONTENT AREA PASSAGES AFFECT STUDENT PERFORMANCE ON 
READING COMPREHENSION TESTS? 

ARNA S. PERETZ 
Ben Gurion University of the Negev, Israel 

Tests of reading comprehension have traditionally been based on 

texts which are considered to be comprehensible to the educated 
layman, thus not requiring specialist knowledge. However, recent 
developments in language testing are moving towards subject-specific 
tests. Such ESP (English for Specific Purposes) tests are based 
on the belief that it is more valid to test the reading compre- 
hension of an engineering student on an engineering topic rather 
than on a social science topic and vice *ersa. Empirical evidence 
in favor of either the general or the specific approach is lacking. 
This paper is a report of a study designed to investigate the 
relevance of student background discipline on tests of reading 
comprehension in EFL (English as a Foreign Language). 185 students 
from three faculties - Science and Technology, Biology, and 
Humanities and Social Science - were tested or three texts related 
to their respective content areas. It was found that content area 
passages do affect student performance on reading 'comprehension 
tests, but not as greatly as had been expected. It was also found 
that the order of presentation of a reading passage in a multiple- 
text test does not affect student performance. 
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More and more university students throughout the world are required 
to learn EFL (English as a Foreign Language) in order to read 
textbooks and professional journals which are written in English. 
This in turn leads to the need to develop tests which permit an 
effective assessment of the reading comprehension abilities of 
students who come from a wide range of academic disciplines, but 
who are often grouped together in broad categories in EFL instructional 
courses. 

It follows then, that a pertinent question for test constructors is 
whether reading comprehension ability is assessed better by tests 
whose content is subject specific, i.e. related to the students 1 
general field of study, or by tests whose content is more global, 
i.e. of general interest. 

Proponents of the general-interest text in reading comprehension 
tests argue that reading proficiency, end not subject-related 
knowledge, should be tested. Therefore, a particular group of 
studenta is not favored when the text is of general interest. 
Furthermore, if students can demonstrate comprehension on a general- 
interest text, it can be assumed that they can comprehend texts in 
their own field of study. 

V 

fc 

These arguments are challenged by Alderson and Urquhart who have 
stated that "it is arguable that there is no such thing as a general 
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text,. •since what is general knowledge for one reader may be highly 
specific and esoteric for another. This is particularly the case 
with students from quite different cultures" (1983:121) . They also 
note that 

uib may be hesitant about accepting the existence of a 
general reading ability. While such a thing may cer- 
tainly exist among educated native speakers who are 
exposed to a wide variety of texts, it is quite 
feasible to suggest that an L2 learner mey have 
acquired much more specific skills for dealing with a 
far narrower range of text types (1983:121-122), 

Recent developments, particularly in the United Kingdom, have led 
to the development of tests which are more specifically suited to 
the target discipline of the test takers. However, this trend 
"merely reflects changes in language teaching practice towards 
specific purpose teaching, ESP" (Alderson end Urquhart 1983:122), 
Empirical evidence on the effect of subject-specific related texts 
to performance on reading comprehension tests is extremely limited. 

Those in favor of the subject-related text in reading comprehension 
tests point out that it seems more appropriate to test students on 
texts which are related to their field of study, especially in an 
ESP context, A student "may have acquired a competence in reading 



texts related to his subject area without being able to displny 
this competence towards 'general texts 111 (Alderson and Urquhart 
1983:122). 

Arguments can be presented against the subject-related text in 
reading comprehension tests, When students from a number of 
related disciplines are grouped together, a subject-specific text 
may favor a particular group. Moreover, it may be difficult to 
"fit" cross-disciplinary students with e suiteble subject-specific 
text. Finally, practical and financial considerations may preclude 
the construction of a large number of tests. 

REVIEW OF THE LITERATURE 

The effect of content familiarity, vocabulary clues, and English 
proficience on clozs scores of 312 students grouped in three broad 
categories - Arts, Sciences, and Social Sciences - at Chinese 
University of Hong Hong was investigated by Moy (1975). While 
significant interaction between academic field of subject end 
content matter of test passage was found, Moy claims that "the 
actual source of the interaction is difficult to explgin* (1975:91). 
Although Science majors in this study obtained highest mean scores 
on the science passage, they elso had highest mean scores on the 
other passages* The Social Science majors had highest mean scores 
on the history passage on which it uas predicted that the Arts 



majors would do best, Moy concluded that the "original hypothesis 
that majors would do better on cloze passages dealing with their 
own academic fields was not demonstrated in this study 11 (1975:71), 
Moy (1975) emphasized both the importance and the difficulty of 
controlling for passage difficulty across content erea (the Dala 
Chall readability formula was found to be an insufficient measure) 
as well as for language proficiency ecross faculties* 

Somewhat different results were obtained by Alderson and Urquhart 
(1983) in a small pilot study conducted with 37 graduate students 
from different academic disciplines and from verious language 
backgrounds. Subjects were grouped into four categories and tested 
on cIozb passages related to three content areas. Language 
proficiency waa controlled to some BxtBnt, but passage difficulty 
was determined on the basis of the results of the test. On the 
whole, it was found that students did better on texts taken from 
their own subject discipline than did students from other disciplines 
on the same text. HowBVBr, text difficulty seems to have outweighed 
content familiarity in this atudy. Thus f all students, including 
the Arts and Social Science students, found the general text to be 
the most difficult and one of the engineering texts to-be the easiest. 

It should be noted that both Alderson and Urquhart (1983) and Mcy 
(1975) used the cloze test in their studies* It is questionable 
whether the cloze format is the best way to assess the kind of 
reading comprehension required for reading academic textbooks and 
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professional journals.- The filling-in of specific blanks is 
predicated more on a knowledge of vocabulary, syntax, and grammar 
than on comprehension and understanding of the subject and of the 
author's purpose and viewpoint. 

Lipson's (198*0 subjects were fourth, fifth, and sixth graders, 
designated as above average readers, who were assigned three 
reading passages, the contents of which were assumed to be totally 
familiar, partially familiar, and totally unfamiliar. On retrieval- 
of-inf ormation tasks, subjects scored highest on the passage with 
the totally familiar content and lowest cn the passage with the 
partially familiar content. This finding accords with the results 
of previous studies (Reynolds et al. 1982; Steffensen at al. 1979) 
which suggest that in some cases prior knowledge, even among fluent 
readers, can interfere with comprehension, if it conflicts with 
information in the text. 

RESEARCH QUESTIONS 

This paper describes a study conducted at Ben Burion University of 
the Negev, Beer Sheva, Israel, in the spring of 198**. 'The study 
investigated the relevance of subject-specific reading passages to 
performance on reading comprehension tests for advanced EFL student9. 
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The research questions were formulated ss follows: 

1. Will students of EFL perform better on a reading comprehen- 
sion tests whose reading content is related to their general field 
of study, or on a reading comprehansion test whose content is 
related tD another subject, given that the texts are of approxi- 
mately comparable difficulty and comprehensible to the educated 
layman? 

2, Dobs the position r# : £ text in a multiple-text exam affect 
reading comprehension perfoif^n^? 

METHODOLOGY 

The subjects consisted of 1B5 EFL students at Ben Gurion University 
of the IMegev: 107 students in four Science and Technology classes, 
29 students in the single Biology class, and U9 students in three 
Humanities and Social Science classes, (At Ben Gurion University 
of the IMegev students of EFL from related disciplines are grouped 
into the aforementioned categories, Separete readers,, with reading 
material related to the background disciplines of the students, are 
used for instruction by each group*) Since students are pieced 
into one of the various instructional levels of EFL - Basic I and II, 
Intermediate I and II; and Advanced I and II - on the basis of a 
centrally administered nationwide EFL placement exam, it was 
assumed that the subjects of this study (Advamced II students) 
were all at approximately the same level of reading proficiency. 

Three passages were excerpted from articles that had appeared in 



Advanced II Science and Technology, Biology, and Humanities and 
Social Science readers; the students were not familiar uith the 
passages. The Humanities and Social Science passage dealt uith 
the ethics of experimentation uith human subjects; the Biology 
passage uith toxic substances and ecological cycles; and the 
Science and Technology passage uith the fuel Bconomy of light 
vehicles* All the paseages uere approximately the same length. 

The format of the test was similar to tests administered during the 
school year* Each test passages uas folloued by a set of questions. 
The questions uere grpuped into three broad categories: general 
comprehension, referent, and vocabulary-in-context. The three 
formsa of the test uere obtained by changing the order of the test 
passages and thBlr related questions (Form I: Biology, Science/Tech- 
nology, Humanities/Social Science; Form II: Science/Technology , 
Humanities/Social Science, Biology; Form III: Humanities/Social 
Science, Biology, Science/Technology), ThB three different test 
forms uere randomly distributed to each class that participated in 
the study (one-third received Form I, one-third Form II, and one- 
third' Form III). 

The test uas administered during the students 9 regular class hours 
and students uere alloued Vk hours to complete the test. No 
dictionaries uere permitted to be used. 



RESULTS 



The means, l\Ps f and standard deviations of the scores for each content 
area test passage are presented in Table 1. The mean scores (per- 
centages) for the entire test for each faculty are presented in 
column four and the mean scores for each test passage across 
faculties are presented at the bottom of the table. 

— - Insert Table 1. 

A breakdown of the results by faculty showing the number of students 
that performed best on each subtest is presented in Table 2% 

Insert Table 2. 

A three-way analysis of variance (three versions, three content 
area texts, three faculties) yielded significant differences in 
the overall performance between students of the three faculties 
|jF f (2 # 175«2B # 37)p<VDDD3l1 and a significant interaction between 
text and faculty [Ff U-35D*l<4.92)p<.0DQl) • 

A one-way comparison of variances between different texts in each 
faculty showed no differences for text3 for students of Humanities 
and Social Sciences, while differences were significant for students 
of Biology \f$ 2 # 5GeG # Gl)p< # DD3j and for students of Science and 
Technology [F f <2.216sfc9 # 9fc)p<.000l| „ 



Table 1. Means, n f s f and standard deviations of scores for each 
content area text, classified by faculty 





Cent 


ent Area 


Mean for 
faculty 




Humanities/ 
Social Science 


BiDlDgy 


Science/ 
Technology 


X=5^.2D 
n=i»9 


Humanities/ 
Social Sci- 
ence Faculty 

n= k3 


x\s57.G5 

S.D.=20.06 

n=l»9 


X\*5D.7G 
S D =21 7 
n=i»9 


X.51.89 

S D «?□ 13 
n=49 


□lDiagy 
Faculty 

n=Z3 


X=58.S2 
S.D»=21 # 15 

n=29 


XV7l».5G 
S.D.=17.8 

n=29 


X=63.D5 
S.D.=20.38 

na29 


X«65.t»0 
n=29 


Science/ 

Technology 

Faculty. 

n=107 


jUsi.SG v 

S.D.=18.7 

n=107 


X«72„19 

S.D r =ia.5 

n=107 


X=82.77 
S.D.a20.2 

n=rlD7 


X=72.17 
n=107 


Mean for 
content area 
teat passage 


X=60.06 

S*D D=5 19.£»3 

N.I85 


X-GG.89 

S.D.=21.53 

N-185 


X«=71.5D 
S.D.=2^.35 

N=185 





9b 



Table 2 # Percentage of students who performed best on subtests, 
classified by faculty 



Highest score 


Highest score 


Highest score 


Highest score 


Scored 


by text 


on Sumanities/ 


on Biology 


on science/ 


equally 


Social science 


text 


Technology 


high on 


? aculty ' 


text 




text 


2 or 3 








texts 


fumanities/ 


38.7 96 


22.596 


16.396 


22.5% 


Social Science 


n=19 


n=ll 


n=8 


n=ll 












Jiology 


2k.l% 


5596 


3% 


10.396 




n=7 


n=16 


n=l 


n=5 


M= 29 










Science/ 


B.k% 


2k. 3% 


39.3% 


2896 


Technology 


n=9. 


n=26 


n«*»2 


n=3D 


\I=1D7 
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A Schaff £ post hoc test indicated that students of Biology did 
significantly better (p<*01) on the Biology test than on the other 
two tests for which there was no significant difference, while 
for the students of Science and Technology all differences were 
significant at the p<»Ql level. 

Because a marginal significance for text by version [F, (4.350« 
2.56)p< # 0^ was obtained, three one-way analyses of variance 
comparing various positions of each test passage were performed. 
These did not yield significant differences. 

Test reliability (Kuder Richardson formula 20^.73) proved 

* 

acceptable, though not very high. The small number of test items 
and the fact that they test different skills may account for this 
in part* 

DISCUSSION 

Expectations that the mean scores for the entire test would ba 
approximately the same for students of the three faculties but 
significantly higher on the f ield-of-study related test passage 
for students of each faculty were only partially realized. 

As indicated by the figures in Table 1, students of Science and 
Technology obtained the highest mean grade *r.n the entire test as 
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well as the highest mean grades on all individual test passages 
(with the exception of the Biology-related test passage whera they 
scored only slightly lower than the Biology students) . One possible 
explanation for the fact that Science and Technology students did 
bo wbII on the entirB test might be related to the fact that the 
entrance requirements of the faculties of Science and Technology 
are higher than those of the faculty of Humanities and S Dc i a i 
Science. The higher mean scores of the Science and Technology 
students could f therefore, reflect a higher level of overall 
competence* 

Another explanation for the better performance of the Science and 
Technology students on the entire test might be a result of the 
particular comprehension strategies developed by this group of 
students. Science and Technology students are taught how to 
comprehend processes, hypotheses, theories, experiments, etc, 
whereas Humanities end Social Science students usually read in 
order to determine author f s main idea, supporting ideas, and 
general viewpoint. This difference in approach to the reading 
material might result in the development of more effective reading 
comprehension strategies. (Moy (1975) also found that-Science 
majors scored higher than Social Science and Arts majors.) 

The question whether students would perform better on *a reading 
comprehension test if- the content of the reading passage were 
related to their general field of study than if the reading 
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passage were related to another subject was not answered conclusively 
While there was a statistically significant difference in performance 
on subject-related test passages for students of Science and 
Technology and for students of Biology, the Humanities and Social 
Science students did not do significantly better on the test passage 
that ubs considered to be more closely related to their academic 
disciplines. 

In a more detailed analysis of the data, the results obtained by 
each student on the three test passages were ranked and the 
percentage of students in each faculty who obtained the highest 
score on their respective f ield-of-study related passage uias noted. 
The results (presented in Table 2) indicate that only in the case 
of the Biology students did more than half (55%) receive highest 
scores on their subject-related test passage. For Humanities and 
Social Science students and for Science and Technology students 
this percentage decreased to ebout 39%. In other words, for most 
of the students in this study, the fact that a text in a reading 
comprehension test was related to their general field of study did 
not appear to make it- easier to comprehend than texts related to 
other subjects. 

□ne possible explanation for this finding is that the texts selected 
for the study were only indirectly related to the students 1 
specialized fielde of study, and thus the content may have been 
only pertially familiar. If this conjecture is true, the results 
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obtained in this study-would support Lipson's (1983) conclusions 
that a totally unfamiliar text is often easiar to comprehend than 
a text with a partially familiar content. In any case, the 
findings obtained in this study indicate that just because a text 
is in the general area of a test taker's specialized field of 
study, it is not easier to comprehend than a text on a, more general 
topic. 

There are two possible explanations for the fact that there was no 
statistically significant interaction between text and faculty for 
students of Humanities and Social Science. The Humanities end 
Social Science-related, test passage was less specifically related 
to the subji. .ft 1 beckground disciplines and thus conferred less of 
an advantage on the Humanities and Social Science students. A 
second explanation mey- ba that since the Humenities and Social 
Science students were the poorest performers on the entire test, 
it is possible that relevance of subject-related content matter 
to performance on a reading comprehension test is related to level 
of reading competence. In other words, one could ask who will 
profit more from the fact that a reading passage is on a familiar 
subject - a more competent reader or a less able one? The findings 
of this study seem to suggest that the more proficient readers (in 
this study, the Science and Technology and the Biology students) 
were those who benefited most from content-related reading passages 
on the reading comprehension test, while the less proficient readers 
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(in this study f the Humanities and Social Science students) 
benefited the least, 'This conjecture needs to be carefully 
investigated , however. 

i 

It was also found that different forms of the same test, constructed 
by changing the order of presentation of text passages and related 
questions, did not affect performance on the test. 

CONCLUSION 

It appears that student performance on reading comprehension tests 
was affected in some cases by content-related passages, but less 
than had been expected. Bas&d on the findings, it may be argued 
that texts which are only indirectly related to the test taker's 
major field of study do not greatly change performance on reading 
tests. It could be that reading competence in EFL may be a factor 
in determining whether content-related passages are relevant to 
performance on reading comprehension tests. Although it seems that 
a text which deals with ethical concerns rather than with the fuel 
economy of cars is more appropriate to test the reading comprehension 
of Humanities students, the findings indicate that the^effect on 
the scores is minimal. Therefore, in situations where students 
from different disciplines are grouped in broad categories, such 

i 

es Science and Technology or Humanities and Social Science, the 
construction and administration of different content-related tests 
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may not be justified. 

Although order of presentation of test passages does not seem to 
affect performance, this finding has practical implications where 
it is desirable to have different forms of the same test.. 

SUGGESTIONS FDR FURTHER RESEARCH 

Suggestions for further research include replicating the study 
using reading passages which are more specifically related to the 
students 1 specialized fields of study. It is also suggested that 
a language test be administered so that the relationship between 
language proficiency and performance on subject-related passages 
may be investigated. This study could be repeated using only 
subjects from a single background discipline; in this way familiarity 
with content can be better controlled. 
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